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@ Sensing istyius position using single 1-D imge sensor. 



(57) An apparatus and method for determining ttie 
location of an object in an active area of a first 
plane. The apparatus includes a first reflecting 
device, disposed substantially perpendicular to 
tiie first plane at a periphery of ttie active area. 
The first reflecting device receives a first image 
of the object from the active area and reflects 
the first image back toward the active area 
substantially parallel to the first plane. A second 
reflecting device, disposed substantially per- 
pendicular to the first plane at a periphery of tiie 
active area, receives a second image of the 
object from ttie active area and reflects the 
second image back toward the active area sub- 
stantially parallel to tiie first plane. The second 
reflecting device is disposed at a first angle 
which is less than one-hundred eighty degrees 
to the first reflecting device. The angle opens 
toward ttie active area and toward a detecting 
device. The detecting device is disposed in ttie 
first plane at a periphery of tiie active area 
opposite ttie first and second reflecting devices. 
The detecting device receives tfie first Image 
and tiie second image and produces a signal 
indicating the position of the first and second 
images. 
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Field of the Invention 

The present Invention relates to sensing the pos- 
ition of an object In a plane and, more specifically, 
sensing the position of a stylus on a surface using a 
single camera and two mirrors. 

Related Art 

Users of pen-based computers generally input 
data into a computer by writing with a special elec- 
tronic stylus directly on the computer's liquid-crystal 
screen, rather than by typing on a keyboard. Program 
choices can be made by checking boxes on the 
screen, as if one is filling out an electronic form. 
Documents can be edited by making movements with 
the stylus on the screen, such as crossing out a word. 
Diagrams can be drawn directly on the screen. 

When designing a pen-based computer system a 
significant consideration is detennining the position 
of the pen or stylus. Many different techniques have 
developed to detect the stylus position. However, sig- 
nificant drawbacks still exist. The sensing (detecting) 
technology must be lightweight, consume little elec- 
tronic power, be durable and withstand electromag- 
netic interference from other parts of the computer. 

Moreover, capturing handwriting requires the 
system to be accurate to within, for example, five 
thousandths of an inch. This level of accuracy is far 
more difficult to achieve than that required in other 
applications such as determining whether a finger is 
pointing to the "deposit" box or the "withdrawal" box 
on the touch screen of an automated teller machine. 
If the pen-sensing system makes even small errore. 
the computer might not be able to make fine distinc- 
tions, such as discriminating the letter "Z" from the 
number "2," no matter how proficient the computer's 
handwriting-recognition software is. Without a highly 
precise mechanism for sensing the position of the sty- 
lus these computer systems will not reliably and ac- 
curately function. 

In addition to reliability and accuracy, the position 
sensing mechanism must be cost effective, durable 
and, easy to use. Some technologies, for instance, re- 
quire the stylus to be connected to the computer by a 
wire, while others allow for a stylus to be detached. 
In general, it is desirable for a stylus to be detachable 
from the computer and the sensing device in order to 
improve the system's flexibility. However, the cost of 
manufacturing the stylus should not be excessive 
since a detachable stylus may be easily misplaced. 

In addition, some previous systems require the 
stylus to touch the screen in order to be detected. 
Other systems can detect the stylus when it is merely 
near the screen. It is desirable to have the ability to 
sense the stylus before it touches the screen in order 
to help the computer system provide instant feed- 
back. 



There are three basic pen-sensing technologies. 
One is a resistive coating technique in which the front 
of a screen is coated with indium-tin oxide, a transpar- 
ent material that conducts electricity. The voltage in 
5 this coating is made to vary gradually from the top to 
the bottom of the screen and from the left to the right 
of the screen. When the stylus contacts the coating, 
an electrical current flows through the stylus and is 
transmitted to the computer via a wire. Since the vol- 
10 tage varies across the screen, the amount of electri- 
city flowing through the stylus will vary depending on 
where it touches the screen. This technology Is inex- 
pensive. However, this technology requires the stylus 
to be wired to the computer and to make contact with 
IS the soreen in order for the stylus position to be detect- 
ed. In additton. since the coating is on the front of the 
screen, it is subject tp wear. 

A second approach uses electromagnetic waves, 
essentially a faint radk) transmission from the stylus 
20 to the computer or vtee versa. A grid of wires behind 
the screen emits a signal whk:h causes a circuit in the 
stylus, which is tuned to the signal frequency, to res- 
onate and transmit a signal of its own. The computer 
then stops transmitting and listens for the signal from 
25 the stylus. The position of the stylus is detenmined by 
the posltton of the wires in the grid which detect the 
signal emanating from the circuit in the stylus. 

This second approach can sense the stylus loca- 
tion before it touches the screen. In additton. the sty- 
30 lus need not be connected to the tx)mputer by a wire 
and may be an unpowered. passive circuit However, 
this system is expensive, with a stylus costing $80 or 
more. In addition, the grid of wires behind the screen 
make it more difficult for screens to be bacWiL There- 
35 fore, screens using this system are typically more dif- 
ficult to read. 

A third approach is similar to the electromagnetic 
one, i.e., the second approach, but senses by electro- 
static coupling rather than electromagnetic coupling. 
40 This system can also detect the stylus without requir- 
ing the stylus to conted the screen. However, the sty- 
lus must be actwe and therefore must either be wired 
to the computer or contain a battery. Styluses using 
this approach are less costly than those in the second 
45 approach, but they remain relatively expensive. 

As is seen from the above discussion, in all three 
of the above approaches the drawing surface must be 
specially designed in order to determine the stylus lo- 
cation. That is. in the f irst approach, the drawing sur- 
50 face is coated with an indium-tin oxide. In the second 
and third approaches, a grid of wires must be posi- 
tioned beneath the drawing surface. 

What is needed is a system for accurately sens- 
ing the location of a stylus in a plane. The stylus 
55 should not require connectfon to the computer or to a 
drawing surface. The sensor should be compact while 
not requiring a special surface coating or an embed- 
ded wire grid. The system should be able to sense the 
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position of the stylus before the stylus physically con- 
tacts an object such as a drawing surfece. In fact, a 
drawing surface should not be required to detenmlne 
the stylus position. In addition, the system should be 
durable and inexpensive to build and maintain. 

Summary of the Invention 

The present invention is an apparatus and meth- 
od for detennlnlng the location of an object in an ac- 
tive area of a first plane. The apparatus includes a 
first reflecting device, disposed substantially perpen- 
dicular to the first plane at a periphery of the acth^e 
area. The first reflecting device receives a first image 
of the object from the active area and reflects the first 
image back toward the active area substantially par- 
allel to the first plane. Asecond reflecting device, dis- 
posed substantially perpendicular to the first plane at 
a periphery of the active area, receives a second im- 
age of the object from the active area and reflects the 
second image back toward the active area substan- 
tially parallel to the first plane. The second reflecting 
device Is disposed at a first angle which is less than 
one-hundred eighty degrees to the first reflecting de- 
vice. The angle opens toward the active area and to- 
ward a detecting device. The detecting device is dis- 
posed in the first plane at a periphery of the acth^e 
area opposite the first and second reflecting devices. 
The detecting device receives the first image and the 
second image and produces a signal Indicating the 
position of the first and second images. 

In the preferred embodiment, the invention Is im- 
plemented in a notebook computer system. The in- 
vention precisely determines the location of the stylus 
without requiring the stylus to be connected to the 
sensing device and without requiring a writing tablet 
or other surface beneath the stylus. 

Brief Description of the Figures 

The foregoing and other objects, features and ad- 
vantages of the invention will be apparent from the 
following more particular description of the preferred 
embodiments of the inventton. as illustrated in the ac- 
companying drawings, wherein: 

FIGURE 1 is a top view diagram illustrating the 
operation of a preferred embodiment of the present 
invention. 

FIGURE 2 is a side view diagram illustrating the 
operation of the preferred embodiment of the present 
invention. 

FIGURE 3 is a top cross-sectional view illustrat- 
ing the operation of a sensor of the present Invention. 

FIGURE 4 is a side cross-sectional view illustrat- 
ing the operation of the sensor of the present inven- 
tion. 

FIGURE 5 is a diagram illustrating the operation 
of an alternate embodiment of the present invention 



having an obtuse angle between the mirrors. 

FIGURE 6 is a diagram illustrating the operatton 
of an alternate embodiment of the present invention 
having an acute angle between the mlrrora. 

5 

Detailed Description of the Preferred 
Embodiments 

A prefenred embodiment of the invention is now 

10 described with reference to the figures, where like raf- 
erence numbers indicate identical or functtonally sim- 
ilar elements. Also in the figures, the left most digit of 
each reference number corresponds to the figure in 
which the reference number is first used. While spo- 
ts clfic steps, configurations and arrangements are dis- 
cussed, it should be underatood that this is done for 
illustrative purposes only. A peraon skilled In the rel- 
evant art will recognize that other steps, configura- 
tions and arrangements can be used without depart- 

20 ing from the spirit and scope of the invention. 

The present invention is an apparatus and meth- 
od for sensing the positbn of an object, e.g., a stylus, 
in a two-dimensional plane using a one-dimensional 
sensing device. In the preferred embodiment, the in- 

25 ventlon is implemented in a notebook computer. In 
this illustrative embodiment, the stylus need not be 
tethered to the computer system. Stylus position can 
be detennined without requiring the stylus to contact 
any portion of the computer system. In addition, a 

30 special stylus tablet is not required. 

Referring first to FIG. 1, the invention includes 
two reflective devices. In a preferred embodiment, 
these reflective devices are mirrore 102, 104, which 
are perpendicularly coupled to each other at one end. 

35 A viewing plane 116 is defined by the drawing sheet 
of FIG. 1. The viewing plane 116 is substantially per- 
pendk:ular to the reflecting surface of the mirrors 102, 
1 04. An active area 1 06 of the system is in the viewing 
plane 116. A stylus 108 is placed into the viewing 

40 plane 116. Numerous images, e.g., between two and 
f ive images depending on the angle between the mir- 
rore 102, 104 (described below), are receh^ed by a 
single image sensing device 110. The image sensing 
device 110 is positioned in the viewing plane 116. Inrv 

45 age sensing device 110 then determines the two di- 
mensional position of the stylus 108 in the viewing 
plane 116 based on the received images. 

The elements of the present invention will now be 
discussed in greater detail. Each min-or 102. 104 has 

50 a reflecting material on the surface facing the active 
area 106 to minimize refraction errors. A mirror hav- 
ing a glass surface.'such that a ray of light must enter 
the glass before being reflected, can be used in the 
present invention. However, errors due to light refrao- 

55 tion will affect the accuracy of the system. Therefore, 
it is preferred that the reflective coating be on the sur- 
face nearest the active area 1 06. 

The mirrore 1 02, 1 04 are perpendicularly coupled 
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such that the tBflecting surfaces are perpendicular to 
the viewing plane 116 defining the active area 106. 

In a preferred embodiment, the reflective surfac- 
es of the mirrors 102. 104 are substantially flat and 
are long enough to ensure that all reflections of a sty- s 
lus 108, positioned in an active area 106, reach the 
sensing device 110. In the prefenred embodiment, the 
height of the mirrors 102, 104 can be small because 
the viewing plane 116 defining the active area 106 is 
perpendicular to the reflecting surface of each mirror. io 

As discussed above, some computer systems 
use a stylus which Is complex and/or expensive. The 
stylus 108 of the present Invention is neither complex 
nor expensive. In the preferred embodiment, a pen- 
light, e.g. Radio Shack catalog number 61 -2538, can 16 
be used as the stylus. Alternate embodiments of the 
stylus Include a pencil or a finger. These embodi- 
ments are discussed below. 

In the preferred embodiment the sensing device 
is a line-scan camera which forms a one-dimensional 
image of the active area 1 06 within the viewing plane 
116. The sensing device 110 will now be discussed 
with reference to FIG. 3 and FIG. 4. The Image sens- 
ing device 110 consists of a projective element 302, 
and a sensing element 304. The projective element 
can be a pinhole or an optical system consisting of 
one or more lenses and/or minws. The projective ele- 
ment forms an optical one-dimensional Image of the 
active area 1 06 within the sensor element 304 by pro- 
jecting a perspective projection from a two-dimen- 
sional plane 116 to a one-dimensional line 402. It is 
prefen-ed that the view angle of the projective element 
302 be at least 90 degrees, to cover a quadrilateral 
formed between the two mirrors and the optical cen- 
ter of the sensing device 110. In the prefenred embodi- 
ment, this quadrilateral, with some modifications de- 
scribed below. Is the active area 106. 

In the preferred embodiment, the sensing device 
110 is a line-scan imaging device, sometimes refer- 
red to as a linear imaging device (LID), for example a 
linear charged coupled device (CCD) array. Its face 
plate contains a linear sensor. I.e., a line of light sens- 
ing elements. A one-dimensional Image is projected 
onto its sensing elements and is converted into ana- 
log electrical signals. 

The analog output of the sensing element is con- 
verted to a digital format within the sensing element 
by a processing device (not shown), or by a separate 
anaiog-to-digital conversion device (not shown). The 
detailed structure and operation of the sensing de- 
vice 110 will be apparent to persons skilled in the rel- 
evant art 

A preferred conf iguratton of the sensing device 
110 consists of a linear CCD sensor 304. for example, 
a Fairchild CCD143A or CCD145, and a projective 
element 302 that projects a one-dimensional image 
at a right angle to the viewing plane 116. When using 
a Fairchild CCD145 linear CCD sensor, the projective 



element 302 would consist of a lens system 310 hav- 
ing a focal iength of approximately 12mm whose opt- 
ical axis lies in the viewing plane 116. 

The sensing device 110 includes a planar mirror 
308 oriented at approximately 45 degrees to the view- 
ing plane 116. This planar mirror 308 projects a one 
dimensional image onto the face plate 402 of the CCD 
sensor 304. Typically, the face plate 402 is parallel to 
the viewing plane 116 and slightly below it. The face 
plate 402 is also orthogonal to the opttoal axis of the 
lens. 

In the preferred embodiment, i.e., when the mir- 
rore 102, 104 are coupled together perpendicularly, 
the sensing device 110 senses four images if the sty- 
lus 108 is in the active area 108 of the viewing plane 
116. The sensing device 110 outputs a vector of elec- 
trically encoded analog or digital values, i.e., vfi], i= 
1 .... N, where N is the number of pixels of the linear 
CCD sensor, and where v[il approximately represents 
the light Intensity received at the optical center of the 
sensing device 110 from direction D[i]. integrated 
over a specific time interval. 

In the preferred embodiment, when a stylus 108 
is placed within the active area 106 four images are 
received by the sensing device 110. Four images are 
received by the sensing device because light emitting 
from the stylus 108 can enter the sensing device via 
four distinct paths. An illustration of the light paths is 
shown in FIG. 1. Light emitting from the stylus entere 
the sensing device 110: (1) directly via path PD. which 
is termed the direct image; (2) by reflecting off mirror 
1 02 via path PR1 , which is termed a single reflected 
image; <3) by reflecting off min-or 104 via path PR2, 
which Is also termed a single reflected image; and (4) 
by reflecting off both mirror 104 and mirror 102 via 
path PRR, which is termed a double reflected image. 

In the prefen-ed embodiment, the sensing devtoe 
110 outputs a vector of digital electrical signals rep- 
resenting a one-dimensional inrtage of the images 
from the two-dimensional active area 106. There are 
numerous methods for determining the two-dimen- 
sionai position of the stylus 108 from this one-dlmen- 
sbnal image. 

A processor (not shown) receives the vector of 
digital electrical signals representing the images .of 
the stylus 108, output from the sensing device 110. 
The processor compares the vector values with a 
fixed threshold value T1 . Pixels having a value above 
this threshold are called Vhite". The "white" pixels 
are represented by a binary "1". Pixels whose value 
is below this threshold are called "black". In the pre- 
ferred embodiment, the white pixels are combined 
into maximally connected regions or runs. A run is a 
continuous chain of white pixels. The length of^he run 
is defined by the number of pbcels in ihe run. Runs 
shorter than a threshold T2 are eliminated. The value 
of T2 depends upon the resolution of the sensing de- 
vice. Each run longer than T2 represents one high- 
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light. The position of a highlight in the 1-D image is 
computed with sut)-pixel accuracy, by fitting a unim- 
odal function, described below, to the values of the 
pixels In the run. The location of a mathematical mode 
of the run is then determined. Typical choices for this 5 
unimodal function are a parabola or a gaussian func- 
tion. It will be apparent to one skilled in the relevant 
art as to how these functions are applied to the pbcel 
values in the run. Using the above process, the proc- 
essor determines a set of highlight positions, each io 
position represented, for example, as a rational num- 
ber ranging from 1 to N (N being equal to the number 
of pixels representing the one-dimensional image of 
the active area 106). 

In Older to determine the position of the stylus is 
108 the light rays received from each image must be 
traced back to the stylus 108. This can be done in 
many ways. One technique for determining the stylus 
position 108 is described below. In the preferred em- 
bodiment four highlights exist, as described above. 20 
Each highlight point is a one-dimensional representa- 
tion of a two-dimensional line in the plane of the ac- 
tive area 1 06. The locations of the mirrors are known. 
Therefore, the reflected path of each ray of light can 
be determined. The position where ail four paths in- 25 
tersect is the stylus location. The preferred technique 
for calculating the stylus position is detailed below. 

To facilitate determining the stylus position, a car- 
tesian coordinate system is chosen to represent the 
plane of the active area 106. The origin of the coor- 30 
dinate system is identified by reference number 112. 
The positive x-axIs extends toward the sensing de- 
vice 110, i.e. to the right, the positive y-axis extends 
along the left mirror, i.e., upwards. The line between 
the sensing device center and the point where the two 35 
mirrors meet will be called the diagonal axis 114. 

A highlight position represents a single ray of light 
entering the sensing device 110. This ray can be rep- 
resented as a 2-D line on the viewing plane 116. The 
equation of this line is obtained from calibration para- 40 
meters of the sensing device 11 0. These parameters 
can be determined from a measurement of the values 
X. y, and u, using procedures which is incorporated 
herein by reference described in the book, O.D. Fau- 
geras, ThfBe-Dimensional Computer Vision, MIT 4$ 
Press, Cambridge, MA, 1992 (restricted to a plane). 
While it will be apparent to one skilled in the relevant 
art how these values are determined, a brief discus- 
sion is provided below. 

The following sensing device model is used. If the so 
position P of the stylus in the active area 108 Is a point 
on the viewing plane 116 with coordinates x and y. 
then the image of P will be a point with position V in 
the 1-D image. The value of u must be between 1 and 
N as defined above. The relationship between, x,y, 55 
and u, i.e.. the location of the stylus 1 08 and the one- 
dimensional image of the stylus, is given by Equation 
[1]: 



(dx + ey + 1)u = ax + by + c [1] 
The values a, b, c. d, and e, in Equation [1] are cali- 
bration parameters of the sensing device 110, which 
are assumed to remain constant In the preferred em- 
bodiment 

If a lens is present in the sensing device 110, the 
relationship between x, y, and u may vary from that 
given in Equation [1] since lenses distort the highlight 
position in the one-dimensional image. This distortion 
may be accounted fbr by replacing u in Equation [1] 
with a non-linear lens distortion function, f(u) which is 
constantfor a given lens. The following analysis as- 
sumes that such a distortk>n function is not required 
to accurately describe the relationship between x, y. 
and u. 

For a given value of u, Equatfon [1] describes a 
line which can be represented as: 

>U + By + C [2] 

where: 

A = utf - a; 

B = ue ' b; and 
C = c - u. 

The line whose equation is given by Equation [2] 
is the line which represents the trace of a light ray en- 
tering the sense array 110. TTie stylus position 108 is 
located on the trace of the direct image PD. discussed 
above, which can be represented by Equation [2]. 

In the preferred embodiment, it is also necessary 
to determine the path of a light ray which enters the 
sensing devtoe 110 after reflecting off one mirror, i.e., 
paths PR1 and PR2 of FIG. 1. 

Given a line equation, LI, describing a ray after 
it Is reflected from a mirror, and a line equation. M, de- 
scribing the locatk)n of the mirror on a plane, then the 
equation of the ray before It is reflected. 1.2. is found 
as follows. Let the reflected ray equation, i.e., the 
equation representing the path of the light ray enter- 
ing the sensing device 120 after being reflected, be 
equal to: 

LI: Ax * By ^ C [3] 
and let the mirror equation be: 

M-.Dx * Ey = F [4] 
The point of reflection [rx,ry] is the intersection of 
lines L1 and M. The coordinates of this point, rx and 
ry, are found by solving the above two equations for 
X and y. 

For simplicity, the following values are defined: 
h = 2{AD + B£)/{D2 + £2) [5] 
Q = - A [6] 
R = /)£ - B [7] 
Using these values, the equation for the ijght ray 
before the reflection. L2, is: 

12: Ox + Ry = Q * rx + R * ry [8] 
The geometry of the mirrors 102. 104 and the 
sensing device 110 are such that the single reflection 
Images, i.e., those whose path is PR1 and PR2 are 
always the right-most and left-most images sensed 
by the sensing device 110. Therefore, Equatton [8] Is 
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used to solve for the path of these two image paths 
PR1 , PR2. The point at which the two paths intersect 
is the stylus position 108. However, in order to mini- 
mize errors, the trace of the other images, i.e.. the di- 
rect image trace PD and the double reflected image 
trace PRR are determined. We can then solve for two 
variables, i.e, x and y, using four equations. A techni- 
que for solving for these equations Is described be- 
low. 

The line defined by Equation [8] represents the 
path of the light ray L2 before reflecting off of a mirror 
and entering the sensing device 110. In the preferred 
embodiment it is also necessary to determine the 
trace of a light ray which enters the sensing device 
110 after reflerting off both minrore 102, 104. i.e., 
pathPRRof FIG. 1. 

As discussed below, which of the two middle im- 
ages is the double reflected image is not immediately 
known. Therefore, in order to detemiine which is the 
double reflected image, each image Is assumed to be 
a double reflected image and the calculations pro- 
ceed under this assumption. The assumption having 
the lowest enror, discussed below, is deemed to be the 
double reflected image. 

If a light ray, LI (whose path is described by 
Equation [3]), which enters the sense device 110, is 
assumed to be a result of a double reflection from two 
minrors, then a line L3 describes this ray before the 
first of the two reflections. The path of L3 is deter- 
mined as follows. Ml represents a line describing the 
position of the left min-or 102. and M2 represents a 
line describing the position of the upper mirror 104. 
The corner point between the two mirrors [cx, cy] is 
the intereection of the lines Ml and M2. This intereec- 
tion can be found by solving the line Equations Ml 
and M2 for x and y. The equation of the trace of the 
light ray entering the sensing device. L1. is: 
L1: Ax + By = C. [3] 

To find L3 the order of the reflections must be de- 
temiined. That Is. It must be determined which mirror 
102, 104 a light ray leaving the stylus 108 first con- 
tacts. With the coordinate system described above, 
parameter "C is never equal to zero if line LI is a dou- 
ble reflection. This is because the origin of the coor- 
dinate system 112 Is outside the active area 106. 
Therefore, if C = 0 we reject the assumption that the 
highlight is the result of a double reflection and stop 
the computation. Otherwise, a test value, T. Is com- 
puted according to Equation [9]. 

r = (A » cx + B » cyVC 19] 

If T < 1 , then the light ray emanating from the sty- 
lus reflectsfirstoff of mirror 102 and then off of minror 
1 04. If T > 1 , then the light ray emanating from the sty- 
lus reflects first off of mirror 1 04 and then off of mirror 
102. If T is approximately equal to 1 then the ray eith- 
er-comes directly from the source, i.e, path PD, or the 
light ray is a double reflection which overlaps with the 
path of the direct ray PD. In both situations the ray is 



treated as if it were a direct ray, PD, and the double 
reflection computation need not continue. 

Once we determine which mirror a light ray emit- 
ted from the stylus 108 first contacts, the path of the 

5 light ray before the first reflection, L3, can be calcu- 
lated by using the technique described above. The 
pre-ref lected path can be determined by first calcu- 
lating the path of the light after a single reflection, us- 
ing Equation [8], and then using this result to calculate 

10 the path of the light before the first reflection. 

In the preferred embodiment, there are always 
two single reflected Images, one direct image, and 
one double reflected image. However, the number of 
identified highlights may be smaller than the number 

IS of images. This occure because within a certain re- 
gion near the diagonal axis 114, the highlight from a 
double reflection will merge with the direct image 
highlight, discussed above. This causes the direct inn- 
age and the double reflected image to appear togeth- 

20 er In the one-dimensional sensor and therefore the 
two images appear as only one biunred image in the 
sensing device 110. if the stylus position 108 Is dose 
to one of the min-ors 102. 104 then a single reflection 
innage may merge with the direct image. This overlap- 

25 ping of the direct image and a single reflected image 
is avoided by separating the active area 1 06 from the 
mirrors 1 02. 1 04 as Is shown In FIG. 1 . 

As discussed below, the highlights conrespond- 
ing to the left-most and right-most rays from the per- 

30 specth^e of the sensing device 110 will be referred to 
as left-most and right-most highlights (images), re- 
spectively. When the double reflected image and the 
direct image overlap, only three highlights are iden- 
tified by the computer. From the perspective of the 

35 sensing device 110, the left-most highlight represents 
the single reflected image which reflects off of mirror 
1 02. The right-most highlight represents the single re- 
flected image which reflects off of mirror 104. The 
middle image is the direct image. The path of the pre- 

40 reflected path is determined for each highlight as dis^ 
cussed at)Ove. 

There are two unknown variables, x and y. and 
there are three equations, one equation for the pre- 
reflected path of each highlight This situation is 

45 termed an overdetemnined system of linear equa- 
tions. Traditionally, such problems are solved by error 
minimization. An ennor of a system of linear equations 
is defined. For example, this error can be defined as 
the sum of squared errors from individual equations, 

so or as the maximum of absolute errors of equations. 
The techniques for solving an overdetemnined system 
of linear equations for minimal error will, be apparent 
to persons skilled in the relevant art. 

In the situation where four highl ights are detected 

55 by the sensing devtoe 110, the left-most highlight and 
the right-most bighlight represent single reflected im- 
ages. Difficulty is encountered In attempting to deter- 
mine which of the two middle highlights represents 

6 
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the direct image and which of the two middle high- 
lights represents the double reflected image. One 
technique for Identifying the double reflected image 
from the direct Image is to create two scenarios, each 
being an overdetermlned system of four linear equa- 
tions L1 , U2, L3, and L4, with two unknowns. The first 
two of these equations are the traced single reflec- 
tions L1 and L2. The remaining two equations are ob- 
tained from the two middle highlights. In the first sce- 
nario, we treat the first of the two middle highlights as 
a direct image. In the second scenario, we treat the 
second of the middle highlights as a direct image. In 
both scenarios, the remaining middle highlight is 
treated as a double reflected image. In each scenario 
an equation is formed to represent the direct ray L3. 
Then the assumed double reflection Is followed to 
find a line, represented by equation L4. The equations 
for each scenario are solved and the stylus position 
is selected as the solution whose associated error is 
smaller. 

Using the above technique, the position of the 
stylus 108, i.e., locations x and y, is determined using 
a single sensing device and two mirrors. Other tech- 
niques for determining the stylus location based on 
the information provided by the sensing device 110 
will be apparent to those skilled In the relevant art 

This technique can be extended to determine the 
po8itk>n of multiple objects in an active area 106. Ad- 
ditionally, the orientation of objects may be detected. 
Another embodiment senses the nrwvement of an ob- 
ject through the active area 106. Such a movement 
can be a finger, or multiple fingers, moving through 
the active area. The computer may be designed to in- 
terpret such a motion, e.g., sweeping a finger through 
the active area may equate to an "erase display" conrv- 
mand. 

The stylus of the present invention need not be 
coupled or tethered to the sensing device 11 0 or to the 
computer (notshown). The present invention does not 
require a light based stylus. A passive stylus such as 
a pen. a pencil or even a finger can be used. When 
using such a passive stylus with ambient light, the 
edges of the active area 106 which are opposite the 
mirrors 102, 104 should be lined with contrasting 
background strips. A good combination is a dark 
background with a light colored stylus, or a stylus with 
a reflective spherical tip to enable a sharper distinc- 
tion between the stylus and the background. 

Another embodiment uses a passive stylus and 
two diffused light sources. The light sources are 
placed along the edges of the active area 106 which 
are opposite the mirrors 102, 104. These light sources 
will form shadows of the stylus which will be detected 
by the sensing device 110. Many opaque items, e.g., 
a pen. a pencil or a finger, can be used as a passive 
stylus. In this situation the computer will determine 
the highlight position based on a run of black pixels 
instead of a run of white pbcels. 



It may be advantageous to reduce the number of 
images the sensing device 110 receive. If the double 
reflected image is eliminated then only three images 
are received by the sensing device 110. The trace of 

5 these three images are immediately known, i.e., the 
left-most image is a single reflected image from mir- 
ror 102, the right-most image is a single reflected im- 
age from mirror 1 04 and the middle image is the direct 
image. The double reflected image may be eliminat- 

10 ed by placing a polarizing filter over each of mirrore 
102, 104 such that the relative polarization angle be- 
tween the two f iltere is ninety degrees. The detailed 
structure and operation of these polarizing filters will 
be apparent to persons skilled in the relevant art 

15 An alternate embodiment of the present inven- 
tion is a techniquefor efficiently determining which of 
the middle images is the double reflected image. Two 
radiation intensity reducing filters are placed adja- 
cent the mirrors 102, 104 or are built Into the mirrors 

20 102, 104. When the sensing devtee senses the im- 
ages, the direct image will have the highest intensity 
and the double reflected image will have the lowest 
intensity. The intensity of the light emitting from the 
stylus is reduced twice, once by each mirror, for the 

25 double reflected image. The sensing device 110 can 
detect the intensity of the radiation it Is receiving. 
Therefore, the middle highlight having the lower inten- 
sity represents the double reflected image. The de- 
tailed structure and operation of the radiation reduc- 

30 ing f Iltere will be apparent to persons skilled In the rel- 
evant art Instead of utilizing radiation intensity reduo- 
ing filters, a similar effect results from using two po- 
larizing filter having a relative polarization angle be- 
tween zero and ninety degrees. 

35 in the preferred embodiment, the mirrore 102, 
104 are positioned such that the reflective surface of 
each mirror 102, 104 is perpendicular to the vievinng 
plane 116. However, in an alternate embodiment, one 
or both mirrors may be positioned such that the 

40 planes which are perpendk^ular to reflective surface 
of each mirror do not align with each other or with the 
viewing plane 116. An example of this can be seen in 
FIG. 2. FIG. 2 represent a side view of the present in- 
vention. I.e., looking directly into mirror 102. 

45 In the preferred embodiment, the viewing plane 
116, which includes the acth^e area 106, is perpendic- 
ular to the reflecting surface of the mirrore 102 and 
104. However, this is not required. For example, in 
FIG. 2 one mirror 102 is aligned according to the pre- 

50 ferred embodiment. i.e., parallel to the Z-axis. How- 
ever, the second mirror 104 is not aligned perpendi- 
cularly to the viewing plane 116, i.e., angle 204 is not 
ninety degrees. As angle 204 diverges from ninety 
degrees the height of the mirrore must increase in or- 

55 der to properly reflect a prevtously reflected image. In 
addition the height 404, in FIG. 4, of the receiving por- 
tion of the sensing device 110, i.e., the projective ele- 
ment 302, must increase to ensure that all images will 
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be properly received by the sensing means 110. 

In the preferred embodiment, the mirrors are per- 
pendicularly coupled in the X-Y plane. In alternate 
embodiments of the present invention, the angle of 
connection, angle 502 in FIG. 5. diverges from ninety 
degrees. When solving for the stylus position 108. the 
min-or angle 602 can vary significantly from ninety 
degrees. In FIG. 5, the min-or connection angle 502 Is 
obtuse. i.e.. greater than ninety degrees. In some in- 
stances having an obtuse mirror angle 502 is an ad- 
vantage over having a perpendicular mirror angle. 
Such an advantageous situation occurs when a direct 
image highlight is merged with a double reflection. 
This occurs when the stylus 108 is positioned on or 
near the diagonal axis 114. When this occurs, the 
sensing device 110 senses a blurred image, dis- 
cussed above. This blurred Image may result in a sig- 
nificant error when determining the stylus position. 

When mirror angle 502 is obtuse, however, no 
such blurring occurs. Therefore, to prevent this type 
of error, the mirrors 102, 104 may be positioned such 
that an obtuse mirror angle 502 is formed. 

The technique for determining the stylus position 
108 when the mirror angle 502 is obtuse is the same 
as that described above in the preferred embodiment. 
In some situations only three images may appear. 
Therefore, In these situations it is necessary to solve 
for the two unknowns, x and y, with three equations. 
A technique for solving this system is described 
above. 

The mirror angle 502 may also be acute, as 
shown in FIG. 6. In this situation a fifth image may be 
detected by the sensing device 110. This fifth image 
results from two images having double reflected trac- 
es. i.e., PRR1 and PRR2. The technique for determin- 
ing the stylus position 108 when the sensing device 
110 receives five images is similar to the technique 
described above when the sensing device 110 re- 
ceives four images. Instead of solving two scenarios, 
three scenarios are solved. It is known that the left- 
most image and the right-most Image are both traces 
of single reflected images. In each of the three sce- 
narios, a different middle image is chosen as the di- 
rect image. Each scenario requires a solution to an 
overdetennined system of linear equations consisting 
of five equations, one for each Image, having two un- 
knowns, X and y. Therefore, as discussed above, 
each of the scenarios will have a solution and an as- 
sociated error. The solution having the smallest asso- 
ciated error is selected as the stylus position. Alterna- 
tively, filters can be used to distinguish double reflec- 
tions, as described above. 

As the minror angle 502 decreases, more than 
five images may be received by the sensing device 
110. These additional Images may be the result of 
double reflections, triple reflections or higher-order 
reflections. The object positbn may be determined 
using the technkiues described above. 



In some situations one or more Images may be 
obscured, for example, if a user positbns a hand on 
a tablet. I.e.. within the viewing plane 116. If such a 
possibility is accounted for, the rule discussed above 

5 stating that the left-most and right-most highlights 
correspond to single reflections can not be used. One 
solution to this problem Is found by forming scenarios 
that allow for some highlights to be obscured, and 
solving these scenarios as shown above. An alterna- 

10 tive is to use f iltere f6r Identification of highlights as 
described above. 

An additk>nal embodiment employs an active sty- 
lus which is equipped with a pressure sensor that re- 
lates light source intensity to the stylus pressure on 

IS the tablet 

While the invention has been particularly shown 
and described with reference to a prefenred embod^ 
ment and several alternate embodiments thereof, it 
will be understood by persons skilled in the relevant 

20 art that various change in form and details may be 
made therein without departing from the spirit and 
scope of the invention. 



25 Claims 

1. An apparatus for determining the location of an 
object in an active area of a f iret plane, compris- 
ing: 

30 first reflecting means, disposed substan- 

tially perpendicular to the f iret plane at a periph- 
ery of the active area, for receiving a f iret Image 
of the object from the active area and for reflect- 
ing said firet image back toward the active area 

35 substantially parallel to the first plane; 

second reflecting means, disposed sut>- 
stanttalty perpendicular to the firet plane at a per- 
iphery of the active area, for receiving a second 
image of the object from the active area and for 

40 reflecting said second image back toward the ac- 
tive area substantially parallel to the firet plane, 
said second reflecting means disposed at a first 
angle which is less than one-hundred eighty de- 
grees to said first reflecting means, said angle 

45 opening toward the active area; and 

detecting means, disposed in the first 
plane at a periphery of the active area opposite 
said first and second reflecting means, for receh^- 
ing said firet image and said second image and 

so for producing a signal indicating the positk>n of 
said first and second images. 

2. The apparatus of claim 1, further comprising 
means for computing the location of the object 

55 within ttie active region based on sakl signal from 
said detecting means. 

3. The apparatus of claim 1 , wherein: 
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said first reflecting means includes a first 
polarization f ilten and 

said second reflecting means Includes a 
second polarization filter; 

wherein a relative polarization angle be- s 
tween said first polarization filter and said sec- 
ond polarization filter is greater than zero de- 
grees, for reducing the radiation Intensity of a 
third image of the object, said third image being 
reflected off of said first reflecting means and io 
saki second reflecting means. 

4. The apparatus of daim 1, wherein said first re- 
flecting means further comprises radiation inten- 
sity reducing means for reducing the radiation in- is 
tensity of said first Image. 

5. The apparatus of daim 4, wherein said detecting 
means further comprises radiation detecting 
means for detecting the radiation intensity of said 20 
first Image for determining if an Image received 

by said detecting means has reflected off one of: 
said first reflecting means, said second reflecting 
means, neither said first reflecting means nor 
said second reflecting means, and both said first 25 
reflecting means and said second reflecting 
means. 

6. The apparatus of daim 1 , wherein the object is a 
radiation emitting stylus, or a passive stylus. 30 

7. The apparatus of daim 1 , wherein said detecting 
means comprises a one-dimensional image 
sensing array for receiving said first image and 

said second image of the object 35 

8. The apparatus of daim 1 , wherein said first angle 
is an acute angle, an obtuse angle, or a right an- 
gle. 

40 

9. A method for determining a location of an object 
in an active area of a first plane, the method com- 
prising: 

(1) reflecting a first image of the object back 

into the active area substantially parallel to 45 
the first plane, from a first reflecting means lo- 
cated at a periphery of the active area; 

(2) reflecting a second image of the object 
back into the active area substantially parallel 

to the first plane, from a second reflecting so 
means located at a periphery of the adive 
area; 

(3) receiving said first image from said first re- 
fleding means and said second image from 
said second reflecting means at a detecting 55 
means located In the first plane across the ac- 
tive area from said first and second reflecting 
means; and 



(4) determining the position of the objed in said 
first plane from said first Image and said second 
Image received at sakl detecting means. 

10. The method of daim 9, further comprising a step 
before step (1) of positbning said first and sec- 
ond reflecting means in the first plane such that 
said first ref leding means defines a first longitu- 
dinal axis substantially parallel to first plane and 
said second refleding means defines a second 
longitudinal axis substantially parallel to the first 
plane whereby said first tongitudinal axis and 
said second longitudinal axis intersect forming 
either an acute angle or an obtuse angle, or a 
right angle facing the iadlve area. 

11. The method of daim 9, further comprising the 
steps, before step (1), of: 

(0.1) positioning a first pdarization filter 
between sakl first reflecting means and the ao- 
th^e area; and 

(0.2) positioning a second pdarization fil- 
ter between said second reflecting means and 
the active area, wherein a relative polarization 
angle between said first polarizatk>n filter and 
said second polarization filter is greater than zero 
degrees, to reduce the radiation intensity of a 
double refleded image detected by saM detect- 
ing means. 

12. The method of daim 9, wherein step (3) further 
comprises the step of recehfing a third, fourth or 
fifth image of the object at said deteding means 
and wherein step (4) when the third image Is re- 
ceived, comprises the step of determining the 
position of the object from at least two of: said re- 
ceived first, second and third images. 

and wherein step (4) when the fourth inv 
age is received comprises the step of determining 
the position of the object from at least two of: said 
received first, second, third, and fourth images, 
and wherein step (4) when the fifth image is re- 
ceived comprises the step of determining the 
position of the object from at least two of: said re- 
ceived first, second, third, fourth, and fifth inn- 
ages. 

13. The method of daim 9, wherein step (3) further 
comprises receiving said first Image and said 
second image on a one dimensional image sens- 
ing anray. 

14. The method of daim 9, further comprising the 
step, before step (1 ) of positioning a first radiation 
redudng means betwe^en said first refleding 
means and said active area to reduce the radia- 
tion intensity of said first image received by sakl 
deteding means. 
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